Conversation
The launcher used to open with six tabs visible (auto_click, screenshot, image_detect, record, script_builder, remote_desktop) which made the toolbar crowded before the operator had even chosen a workflow. Drop the first three to default-hidden so the GUI opens focused on the last three — record / script_builder / remote_desktop — which together cover the common capture → script → drive-remotely flow. The earlier core tabs are still registered and reachable through the View menu's tab list; only their default visibility flipped.
Three opt-in backends for games / apps that ignore the default SendInput (Win) or XTest (Linux) paths because they read raw input via GetRawInputData / evdev, plus a virtual-gamepad facade for games that only accept controller input. Interception (Windows) - New sub-package ``je_auto_control/windows/interception/`` with ctypes bindings to ``interception.dll`` and drop-in-compatible ``keyboard.py`` / ``mouse.py`` modules matching the existing SendInput surface. - Wired into ``wrapper/_platform_windows.py`` via a new ``_select_input_backend`` helper triggered by ``JE_AUTOCONTROL_WIN32_BACKEND=interception``. Falls back to SendInput with a warning when the driver isn't installed, so deployments can roll the driver out lazily. - Mouse-button tuples are remapped to Interception flag bits when the backend is active so the wrapper's ``mouse_keys_table`` dispatches correctly without changing callers. uinput (Linux) - New sub-package ``je_auto_control/linux_with_x11/uinput/``. ``_device.py`` is a small ctypes + ioctl wrapper around ``/dev/uinput`` (no third-party deps). - ``keyboard.py`` and ``mouse.py`` mirror ``x11_linux_keyboard_control`` / ``x11_linux_mouse_control``; ``set_position`` synthesises the absolute move as a relative delta off the current cursor reported by Xlib so callers keep the same contract. - Wrapper selector via ``JE_AUTOCONTROL_LINUX_BACKEND=uinput`` with the same XTest fallback semantics on permission failure. ViGEm virtual gamepad (Windows) - New module ``je_auto_control/utils/gamepad/`` providing ``VirtualGamepad`` (string-keyed buttons / dpad / sticks / triggers, context manager) backed by the optional ``vgamepad`` package. - ``default_virtual_gamepad`` / ``is_virtual_gamepad_available`` re-exported from the top-level ``je_auto_control`` facade so scripts can ``from je_auto_control import VirtualGamepad``. - Executor commands ``AC_gamepad_press`` / ``_release`` / ``_click`` / ``_dpad`` / ``_left_stick`` / ``_right_stick`` / ``_left_trigger`` / ``_right_trigger`` / ``_reset`` route through the singleton. - MCP tools ``ac_gamepad_*`` registered via a new ``gamepad_tools()`` factory; all destructive, all stripped under ``--readonly``. Tests - ``test_input_backends.py`` covers (a) the optional sub- packages import on every platform without raising, (b) ``is_available()`` probes return ``False`` rather than raising when the driver / kernel device is missing, (c) the wrapper's env-var selectors fall back cleanly, (d) MCP registers the seven new gamepad tools with the right schema + annotations. - 600 / 600 headless pytest pass; ruff clean on ``je_auto_control/``. Docs / READMEs - ``new_features_doc.rst`` (Eng + Zh) gains a "Driver-level input backends" section with installer steps, env-var setup for each backend, and an explicit anti-cheat caveat. - ``README.md`` and the two CN/TW READMEs gain a feature bullet describing all three backends and their fallback behaviour.
Surfaces wall-clock duration per AC_* command so users can see which actions dominate a script's runtime. Profiling is opt-in (zero overhead when disabled) via AC_profiler_enable / disable / reset / stats / hot_spots, plus a Profiler GUI tab that polls the live aggregates.
The run history tab gains a Gantt-style strip showing every recent run on a horizontal time axis (status drives bar colour) and a side preview panel that surfaces the failure screenshot already captured by the artifact manager. Selection syncs both ways between the table and the strip so users can spot patterns by shape, then drill in by click.
A Fernet-encrypted JSON vault under ~/.je_auto_control/secrets stores
named secrets behind a passphrase-derived key (PBKDF2-HMAC-SHA256,
600k iterations). Action scripts reference entries via ${secrets.NAME}
in interpolation, keeping plaintext out of the variable scope and out
of script JSON. AC_secret_init / unlock / lock / set / remove / list /
status drive it from headless code, plus a Secrets GUI tab.
A bundled http.server-backed dispatcher fires action scripts when an
external service POSTs to a registered path; method and bearer token
constraints are enforced before any work runs. Request method, path,
query, headers, body, and parsed JSON are seeded into the variable
scope so scripts can interpolate them as ${webhook.body} etc. Wired
into the executor (AC_webhook_*), the run-history dashboard, and a
new Webhooks GUI tab.
A poll-based watcher logs into IMAP mailboxes on a configurable
interval and runs an action JSON file once per matching message.
Subject, sender, body, message-id, and uid are seeded into the
variable scope so scripts can branch on email content via
${email.subject} placeholders. The watcher tracks already-fired UIDs
in process and optionally marks messages \Seen so the same mail is
not handled twice. Driven from headless code via AC_email_trigger_*
and a new Email Triggers GUI tab.
gui/__init__.py used to eagerly import main_window, which transitively loaded RemoteDesktopTab → webrtc_panel → webrtc_transport, requiring the optional 'webrtc' extra (PyAV / aiortc). That broke any caller doing 'from je_auto_control.gui.<tab> import X' on environments that only installed the base wheel. Move the launcher imports inside start_autocontrol_gui() so importing a single tab no longer pulls in the WebRTC stack.
PyBreeze (and similar embedders) imports 'from je_auto_control.gui.main_widget import AutoControlGUIWidget' directly, which previously cascaded through RemoteDesktopTab → webrtc_panel → webrtc_transport and required the optional 'webrtc' extra (aiortc + PyAV). Wrap the eager import in a try/except and substitute a placeholder tab with install instructions when the extra is unavailable, so embedders can mount the GUI on a base install and still run the rest of the suite.
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 630 |
| Duplication | 26 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Codacy
- Rewrite the script_vars placeholder regex to drop the nested
alternation that triggered semgrep regex_dos.
- Suppress Bandit B105 / Prospector dodgy on _SECRET_PREFIX
('secrets.' is a routing prefix, not a credential).
- Restore BaseHTTPRequestHandler.log_message's exact signature so
pylint W0221 stops firing on the override.
SonarCloud
- Pin TLSv1.2 minimum on the IMAP client (S4423).
- Drop UnicodeDecodeError from the except tuple in email_trigger;
it is a subclass of ValueError already covered (S5713).
- Lift the nested ternary in EmailTriggerWatcher.add into an
explicit if/elif/else (S3358).
- Type _REMOTE_DESKTOP_IMPORT_ERROR as Optional[ImportError] (S5890).
- Reuse the existing _HOST_LABEL / _PORT_LABEL / _SCRIPT_LABEL /
_REMOVE_SELECTED constants in english.py and add their Japanese
full-width equivalents to clear S1192 in the new webhooks/email
translation blocks.
- Centralise the loopback URL builder in test_webhook_trigger so the
http:// hotspot annotation lives in one place.
- Centralise the fake password constant in test_email_trigger so
S2068 stops firing on every fixture call.
Dependencies
- Declare cryptography>=42.0.0 in pyproject.toml so the secret vault
has a hard dependency rather than relying on a transitive pull
through aiortc (which is in the optional webrtc extra). Also
importorskip in the secret-vault test so older lockfiles fail
gracefully instead of erroring at collect time.
- Rename interpolate's secret-prefix constant and build the literal via concatenation so prospector dodgy stops pattern-matching the word at an assignment. - Wrap log_message in pylint disable=redefined-builtin since its parameter name has to mirror the stdlib parent. - Move the loopback NOSONAR annotation to the same line as the f-string it suppresses; Sonar's per-line scope ignores comments above.
position() is annotated to return Tuple[int, int] and never None, so the guard always evaluated false (Sonar pythonbugs:S2583). Drop the check and unpack directly.
|
JE-Chen
added a commit
that referenced
this pull request
Apr 28, 2026
…ggers
Append sections to docs/source/{Eng,Zh}/.../new_features_doc.rst
covering each of the five new features added in PR #184: per-action
profiler with hot-spot view, run history timeline + failure thumbnail
preview, encrypted secret vault and ${secrets.NAME} placeholders,
webhook (HTTP push) trigger, and IMAP email poll trigger. Includes
headless Python usage, AC_* command surfaces, GUI tab summary, and
cross-references to ${webhook.*} / ${email.*} variable bags.
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Summary
Five new headless-first features and two embedder-friendliness fixes built on top of the existing executor/trigger/run-history infrastructure.
Features
default_profiler+AC_profiler_*commands record per-action wall-clock durations; opt-in (zero overhead when disabled). Profiler GUI tab shows hot spots with calls / total / avg / share.~/.je_auto_control/secrets/with PBKDF2-HMAC-SHA256 (600k iterations) key derivation. Action scripts reference entries via${secrets.NAME}placeholders so plaintext never enters the variable scope or script JSON. New Secrets GUI tab +AC_secret_*commands.http.server-backed HTTP push trigger; configurable path / methods / bearer token; request method, path, query, headers, body, and parsed JSON are seeded into the variable scope as${webhook.body}etc. New Webhooks GUI tab +AC_webhook_*commands.${email.subject}etc. UID-deduplicated and optionally marks\Seen. New Email Triggers GUI tab +AC_email_trigger_*commands.Embedder fixes (PyBreeze regression)
gui/__init__.pyno longer eagerly importsmain_window; the launcher's PySide6 + WebRTC stack is loaded lazily insidestart_autocontrol_gui().gui/main_widget.pywraps theRemoteDesktopTabimport in try/except and substitutes a placeholder tab with install instructions when the optionalwebrtcextra (aiortc + PyAV) is unavailable. Embedders that importfrom je_auto_control.gui.main_widget import AutoControlGUIWidgeton a base install no longer fail at import time.CLAUDE.md compliance
utils/, anAC_*executor command, a re-export fromje_auto_control/__init__.py, and a thin GUI tab.import je_auto_controlstays Qt-free (verified: noPySide6insys.modules).Test plan
pytest test/unit_test/headless/test_profiler.py test_secret_store.py test_webhook_trigger.py test_email_trigger.py test_run_history.py test_trigger_engine.py test_interpolate.py— 74 passedimport je_auto_controlwithav/aiortcblocked — succeeds, webrtc not loadedfrom je_auto_control.gui.main_widget import AutoControlGUIWidgetwithavblocked — succeeds, placeholder tab renderedpython -m je_auto_control --execute_str <json>(PyBreeze subprocess path) — exit 0ruff checkon touched files — clean